Understanding of Object Manipulation Actions Using Human Multi-Modal Sensory Data

Object manipulation actions represent an important share of the Activities of Daily Living (ADLs). In this work, we study how to enable service robots to use human multi-modal data to understand object manipulation actions, and how they can recognize such actions when humans perform them during human-robot collaboration tasks. The multi-modal data in this study consists of videos, hand motion data, applied forces as represented by the pressure patterns on the hand, and measurements of the bending of the fingers, collected as human subjects performed manipulation actions. We investigate two different approaches. In the first one, we show that multi-modal signal (motion, finger bending and hand pressure) generated by the action can be decomposed into a set of primitives that can be seen as its building blocks. These primitives are used to define 24 multi-modal primitive features. The primitive features can in turn be used as an abstract representation of the multi-modal signal and employed for action recognition. In the latter approach, the visual features are extracted from the data using a pre-trained image classification deep convolutional neural network. The visual features are subsequently used to train the classifier. We also investigate whether adding data from other modalities produces a statistically significant improvement in the classifier performance. We show that both approaches produce a comparable performance. This implies that image-based methods can successfully recognize human actions during human-robot collaboration. On the other hand, in order to provide training data for the robot so it can learn how to perform object manipulation actions, multi-modal data provides a better alternative

Control of Physical Human-Robot Interaction: Mimicking Human Assistance

Engineering an assistive robot, capable of serving human needs and performing daily chores, has been a long-sought-for goal for the Robotics field as a whole. One of the main challenges facing researchers is on how to build the robot to be accepted by humans. There are many factors involved in having a robot and a human effectively collaborating, including technological limitations, anthropomorphic elements, ethical concerns, social factors, etc. One of the less explored aspects of this problem is physical interaction between a human and a robot. Envision a robotic assistant that is helping a human, moving a piece of furniture. Since the human and the robot are haptically coupled, every small movement/force of the robot is perceived by the human and can be interpreted as a clue for the next action. At the same time, the human expects the robot to understand the cues he/she is giving. In other words, the human expects the interaction to be fluid and natural, as it is with a human partner. Note that in a physical interaction between two humans, the kinesthetic cues serve as a communication channel that guarantees the success of the collaboration, even in cases when the verbal communication is missing. In this thesis, we focus on the physical interaction between a human and a robot. We first study the characteristics of a natural human-human physical interaction and explore different features of cooperation between two humans. In particular, we propose an abstract model for the quality of cooperation, a mathematical model for the motion trajectory during the interaction and a novel approach in modeling the interaction force between two humans. Based on these models that we construct for a natural human-human interaction, we propose a set of control policies that replicates the same interaction features and mimics human’s behavior during a physical interaction between a human and a robot

A Model for Human–Human Collaborative Object Manipulation and Its Application to Human–Robot Interaction

During collaborative object manipulation, the interaction forces provide a communication channel through which humans coordinate their actions. In order for the robots to engage in physical collaboration with humans, it is necessary to understand this coordination process. Unfortunately, there is no intrinsic way to define the interaction forces. In this study, we propose a model that allows us to compute the interaction force during a dyadic cooperative object manipulation task. The model is derived directly from the existing theories on human arm movements. The results of a user study with 22 human subjects prove the validity of the proposed model. The model is then embedded in a control strategy that enables the robot to engage in a cooperative task with a human. The performance evaluation of the controller through simulation shows that the control strategy is a promising candidate for a cooperative human-robot interaction

Ethnic differences in the lifestyle behaviors and premature coronary artery disease: a multi-center study

Background: Diverse ethnic groups that exist in Iran may differ regarding the risk factors such as hypertension, hyperlipidemia, dyslipidemia, diabetes mellitus, and family history of non-communicable disease. Premature Coronary Artery Disease (PCAD) is more endemic in Iran than before. This study sought to assess the association between ethnicity and lifestyle behaviors in eight major Iranian ethnic groups with PCAD. Methods: In this study, 2863 patients aged ≤ 70 for women and ≤ 60 for men who underwent coronary angiography were recruited in a multi-center framework. All the patients’ demographic, laboratory, clinical, and risk factor data were retrieved. Eight large ethnicities in Iran, including the Farses, the Kurds, the Turks, the Gilaks, the Arabs, the Lors, the Qashqai, and the Bakhtiari were evaluated for PCAD. Different lifestyle components and having PCAD were compared among the ethnical groups using multivariable modeling. Results: The mean age of the 2863 patients participated was 55.66 ± 7.70 years. The Fars ethnicity with 1654 people, was the most subject in this study. Family history of more than three chronic diseases (1279 (44.7%) was the most common risk factor. The Turk ethnic group had the highest prevalence of ≥ 3 simultaneous lifestyle-related risk factors (24.3%), and the Bakhtiari ethnic group had the highest prevalence of no lifestyle-related risk factors (20.9%). Adjusted models showed that having all three abnormal lifestyle components increased the risk of PCAD (OR = 2.28, 95% CI: 1.04–1.06). The Arabs had the most chance of getting PCAD among other ethnicities (OR = 2.26, 95%CI: 1.40–3.65). While, the Kurds with a healthy lifestyle showed the lowest chance of getting PCAD (OR = 1.96, 95%CI: 1.05–3.67)). Conclusions: This study found there was heterogeneity in having PACD and a diverse distribution in its well-known traditional lifestyle-related risk factors among major Iranian ethnic groups